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Abstract — We study the generation of a secret key of maximum 
rate by a pair of terminals observing correlated sources and 
with the means to communicate over a noiseless public com- 
munication channel. Our main result establishes a structural 
equivalence between the generation of a maximum rate secret 
key and the generation of a common randomness that renders 
the observations of the two terminals conditionally independent. 
The minimum rate of such common randomness, termed in- 
teractive common information, is related to Wyner's notion of 
common information, and serves to characterize the minimum 
rate of interactive public communication required to generate an 
optimum rate secret key. This characterization yields a single- 
letter expression for the aforementioned communication rate 
when the number of rounds of interaction are bounded. An 
application of our results shows that interaction does not reduce 
this rate for binary symmetric sources. Further, we provide an 
example for which interaction does reduce the minimum rate of 
communication. Also, certain invariance properties of common 
information quantities are established that may be of independent 
interest. 

Index Terms — Common information, common randomness, in- 
teractive communication, interactive common information, secret 
key capacity. 

I. Introduction 

Consider secret key (SK) generation by a pair of terminals 
that observe independent and identically distributed (i.i.d.) 
repetitions of two discrete, finite-valued random variables (rvs) 
of known joint probability mass function. The terminals com- 
municate over a noiseless public channel of unlimited capacity, 
interactively in multiple rounds, to agree upon the value of 
the key. The key is required to be (almost) independent of 
the public communication. The maximum rate of such an SK, 
termed the secret key capacity, was characterized in llTjI . (HI. 

In the works of Maurer and Ahlswede-Csiszar ifTSl . [I], 
SK generation of maximum rate entailed both the terminals 
recovering the observations of one of the terminals, using the 
least rate of communication required to do so. Later, it was 
shown by Csiszar-Narayan fS] that a maximum rate SK can 
be generated also by the terminals recovering the observations 
of both the terminals. Clearly, the latter scheme requires more 
communication than the former In this paper, we address the 
following question, which was raised in |5j Section VI]: 
What is the minimum overall rate of interactive communication 
RsK required to establish a maximum rate SK? 
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We answer this question by characterizing the form of 
common randomness (CR) (i.e., shared bits, see ||2]) that the 
terminals must establish in order to generate a maximum 
rate SK; two examples of such common randomness are 
the observations of any one terminal |[T3l . lH] and of both 
terminals |!5l. While our main result does not yield a single- 
letter characterization, it nonetheless reveals a central link 
between secrecy generation and Wyner's notion of common 
information (CI) between two dependent rvs X and Y lfT6l . 
Wyner defined CI as the minimum rate of a function of 
i.i.d. repetitions of two correlated random variables X and 
Y that facilitated a certain distributed source coding task. 
Alternatively, it can be defined as the minimum rate of a 
function of i.i.d. repetitions of X and Y such that, conditioned 
on this function, the i.i.d. sequences are (almost) independent; 
this definition, though not stated explicitly in lfT6l . follows 
from the analysis therein. We introduce a variant of this notion 
of CI called the interactive CI where we seek the minimum 
rate of CR that renders the mentioned sequences conditionally 
independent. Clearly, interactive CI cannot be smaller than 
Wyner's CI, and can exceed it. Our main contribution is 
to show a one-to-one correspondence between such CR and 
the CR established for generating an optimum rate SK. This 
correspondence is used to characterize the minimum rate of 
communication Rsk required for generating a maximum rate 
SK. In fact, it is shown that Rsk is simply interactive CI 
minus the secret key capacity. 

When the number of rounds of interaction are bounded, 
this characterization yields a single-letter expression for Rsk- 
Using this expression we show that an interactive communi- 
cation scheme can have less rate than a noninteractive one, in 
general. However, interaction offers no advantage for binary 
symmetric sources. This expression also illustrates the role 
of sufficient statistics in SK generation. We further dwell 
on this relationship and show that many CI quantities of 
interest remain unchanged if the sources are replaced by their 
corresponding sufficient statistics (with respect to each other). 
Interestingly, the effect of substitution by sufficient statistics 
has been studied in the context of the rate-distortion problem 
for a remote source in jT] Lemma 2], and recently, for the 
lossy and lossless distributed source coding problems in ifTTl . 
Here, in effect, we study this substitution for the distributed 
source coding problems underlying the CI quantities. 

The basic notions of CR and SK are explained in the next 
section. The definition of interactive CI and the heuristics 
underlying our approach are given in Section |III] Our main 
results are provided in Section |IV] followed by illustrative 
examples in the subsequent section. Section [VT] explores the 
connection between sufficient statistics and common infor- 
mation quantities. A discussion of our results and possible 



extensions is given in the final section. 

Notation. The rvs X and Y take values in finite sets X and y, 
respectively. Let X" = (Xi,...,X„) and F" = {Yi,...,Yn) 
denote n i.i.d. repetitions of X and Y, respectively. For 
a collection of rvs Ui,...,Ur, for i < j let Uil denote 
Ui,Ui+i...,Uj\ when i = 1, we use U^ = Ui,...,Uj. For 
rvs U, V, and < e < 1, we say U is e-recoverable from V 
if there is a function g of F such that 

P{U = g{V)) > 1-e. 

Denote the cardinality of the range space of a mapping / by 
ll/ll, and similarly, with a slight abuse of notation, the (fixed) 
range space of a random mapping F by ||F||. 

II. Interactive Communication, Common 
Randomness and Secret Keys 

Terminals X and 3^ (with a slight abuse of notation) 
communicate interactively, with, say, terminal X transmit- 
ting first. Each terminal then communicates alternately for 
r rounds. Specifically, an r-interactive communication f = 
(/i: hi ■■■, fr) is a sequence of finite-valued mappings with 

f2^+l : X" X ^2' ^ J-2,+1, < ^ < L(r - 1)/2J , 
/2, : y" X ^2.-1 ^ ^^^^ l<^<Lr/2J, 

where {J^i}l^i are finite sets and Tq = 0. This set-up 
subsumes protocols where terminal y initiates the commu- 
nication upon choosing /i = constant. Let F = f (X",y") 
describe collectively the corresponding rv. The rate of this 
communication is given by 

-log||F||. 
n 

We assume that the communication from each terminal is a 
(deterministic) function of its knowledge. In particular, ran- 
domization is not allowed. This is not a limiting assumption; 
see Section [VII- Al 

Definition 1. Given interactive communication F as above, a 
function L of {X"-,Y"') is e-common randomness (e-CR) re- 
coverable frorrlj F if there exist mappings Li = L]^'{X^, F) 
and La = i^"^(r",F) such that 

F{L = Li=L2)>l-e. 

Definition 2. A function K of (X",F"), with values in 
a set /C, forms an e-secret key for X and Y (e-SK) if K 
is e-CR recoverable from X" or F" and (interactive public 
communication) F, and 



-I{K AF) <e. 
n 



(1) 



For convenience, simplistically, the e-SK K is said to be 
recoverable from F. A rate i? > is an achievable SK rate if 
for every < e < 1 there exists, for some|3 n > 1, an e-SK 

'The rv L is e-recoverable from (X",F) or (y",F) but not necessarily 
from F alone. The deliberate misuse of the terminology "recoverable from 
F" simpUfies presentation. 

^ Our results hold even if the phrase "for some n > 1" is replaced by "for 
all n sufficiently large;" the fornier has been chosen here for convenience. 



K = X^") with {l/n)H{K) >R~e. The supremum of all 
achievable SK rates is denoted by C, and is called the SK 
capacity. 

The following result is well known. 

Tlieorem 1. lfT3l . HI The SK capacity for X and Y is given 
by 

C ^I{X A Y). (2) 

III. Relation between Secret Key and Wyner's 
Common Information 

We interpret Wyner's CI for a pair of rvs {X, Y) as the 
minimum rate of a function of their i.i.d. repetitions (X", F") 
that renders X" and F" conditionally independent. Formally, 

Definition 3. i? > is an achievable CI rate if for every 
< € < 1 there exists an n > 1 and a (finite-valued) rv 
L = L (X", F") of rate {l/n)H{L) < i? + e that satisfies the 
property: 



-I(X" AY'' \L)<e. 
n 



(3) 



Obvious examples of such an rv L are L = (X",y") 
or X" or y". The infimum of all achievable CI rates, 
denoted ClwiX A Y), is called the CI of X and Y. This 
definition of CI, though not stated explicitly in 1161 . follows 
from the analysis therein. The following theorem characterizes 
CIw{X AY). 



Tlieorem 2. pT6| The CI of the rx's X, Y is 

ClwiX AY) ^-aimI{X,Y AW), 



(4) 



where the rv W takes values in a (finite) set W with |VV| < 
I A:" 1 1 3^ I and satisfies the Markov condition X -o-W -o-Y. 

The direct part follows from fTS equation (5.12)]. The proof of 
the converse is straightforward. Further, it is a simple exercise 
to infer from g) that ClwiX AY) > I{X AY). 

Definition 4. An achievable r-interactive CI rate is defined 
in a manner analogous to the achievable CI rate, but with the 
restriction that the rvs L in (O be e-CR, i.e., L = (J, F), where 
F is an r-interactive communication and J is e-recoverable 
from F. The infimum of all achievable r-interactive CI 
rates, denoted CIl(X;Y), is called the r-interactive CI of 
the rvs X and Y. By definition, the nonnegative sequence 
{CIl{X; Y)}^^^ is nonincreasing in r and is bounded below 
by ClwiX AY). Define 

Ch{XAY)^ lim CT[{X;Y). 

r— >-oo 

Then CI,{X A Y) > CIw{X A F) > 0. Note that 
CIl{X\Y) may not be symmetric in X and Y since the 
communication is initiated at terminal X. However, since 

CIl+\X-Y) < CmY;X) < Cir\X-Y), 

^ It is shown in tl4l . (3) that SK capacity remains unchanged even if the 
notion of "weak secrecy" of A" in {T) is tightened to "strong secrecy" by 
omitting the normalization with respect to n, and an additional uniformity 
constraint H{K) > log \K\ — e is imposed. 



clearly, 



CI,{XAY) 



lim CIl{X;Y) 
lim C/[(y;X) 
Ch{Y hX). 



(5) 



Further, for all < e < 1, J = X" is e-recoverable from 
F" and a communication (of a Slepian-Wolf codeword) F ~ 

F (X"), and L = {J,F) satisfies @. Hence, Ch{X A F) < 
i/(X); similarly, CIi{X AY) < H{Y). To summarize, we 
have 

Q<CIw{X AY) <CIi{X AY) < min{i7(X),i?(y)}, 

(6) 

where the first and the last inequalities can be strict. In Section 
IV-AI we show that the second inequality is strict for binary 
symmetric rvs X, Y . 

The r-interactive CI plays a pivotal role in optimum rate 
SK generation. Loosely speaking, our main result asserts the 
following. A CR that satisfies (|5| can be used to generate an 
optimum rate SK and conversely, an optimum rate SK yields 
a CR satisfying (|i|. In fact, such a CR of rate R can be 
recovered from an interactive communication of rate R — C, 
where C is the SK capacity for X and Y. Therefore, to find 
the minimum rate of interactive communication needed to 
generate an optimum rate SK, it is sufficient to characterize 
Ch{XAY). 

IV. Main Results 

Definition 5. A rate i?' > is an achievable r-interactive 
communication rate for CIl if, for all < e < 1, there 
exists, for some n > 1, an 7'-interactive communication F 
of rate (1/n) log ||F|| < R' + e, and an e-CR J recoverable 
from F, with L = (J, F) satisfying (O. Let R^j denote the 
infimum of all achievable r-interactive communication rates 
for CIl- Similarly, R" > is an achievable r-interactive 
communication rate for SK capacity if, for all < e < 1, 
there exists, for some n > 1, an r-interactive communication 
F of rate (1/n) log ||F|| < i?"+e, and an e-SK K, recoverable 
from F, of rate {l/n)H{K) > I{XAY)~e, R^g^ denotes the 
infimum of all achievable r-interactive communication rates 
for SK capacity. Note that by their definitions, both R^(jj and 
R^gj^ are nonincreasing with increasing r, and are bounded 
below by zero. Define 



Rci = lim R 



C/i 



R. 



SK 



lim R 



SK- 



Although R'(jj{X]Y) and i?g^(X;F) are not equal to 
R^Qj{Y]X) and R^gj^{Y]X), respectively, the quantities Rci 
and RsK are symmetric in X and Y using an argument similar 
to the one leading to (|5]l. 

Tlieorem 3. For every r > 1, 

RIk^Rci^CII{X-Y)-I{XAY). (7) 

Corollary. It holds that 

RsK = Rci^Ch{XAY)-I{XAY). (8) 



Remark. The relation (|8]l can be interpreted as follows. Any 
CR J recoverable from (interactive communication) F, with 
L = (J, F) satisfying (|3]l, can be decomposed into two 
mutually independent parts: An SK K of maximum rate and 
the interactive communication F. It follows upon rewriting dHJ 
as CIi{X AY) ^ I{X AY) + Rci that the communication 
F is (approximately) of rate Rci- Furthermore, Rci is same 
as RsK- 

A computable characterization of the operational term 
CIi{X A Y) is not known. However, the next result gives 
a single-letter characterization of CIl{X] Y). 

Theorem 4. Given rvs X, Y and r > 1, we have 

CIl{X;Y)= min /(X, F A J/i, ..., C/^), (9) 

Ul , . . . , U-r 

where the minimum is taken over rvs Ui, ...,Ur taking values 
infinite sets Ui, ...,Ur, respectively, that satisfy the following 
conditions 

(PI) U2^+l -^X,U^'^Y, 0<i< L(r - 1)/2J , 

U2t -^ Y, U^'^^ -^X, l<i< [r/2j , 
(P2) X H^ [/'■ H3- r, 

2?: 
(F3) |iY2.+i| < \X\ l[ \U,\ + 1, < ^ < L(^ - 1)/2J, 



j=i 



2?-l 
3=1 

with Uq ^ 9 and Uq = constant. 

Remark. Note that (|9]l has the same form as the expression for 

CIwiXAY) in @ with W replaced by {Ui, ..., Ur) satisfying 
the conditions above. 

Before presenting the proof of our main Theorems |3] and 
|4] we give some technical results that will constitute central 
tools for the proofs. 

Lemma 5. For an interactive communication F it holds that 
H{F I X") + H{F I r") < H{F). (10) 

Lemma 6. For an r-interactive communication F, define 

f, = f(x;:[,_i)+,,i;:^U)+i), l<^<k. 

Then, for all k > fco(7T,, e, \X\, \y\) there exists an r-interactive 
communication F' = F' (X"'',^"'') of rate 

-^ log IIF'II <-[H (F|X") + H (F|y")] + e, (11) 
nk n 

such that F^ is an e-CR recoverable from F' . 

Remark. Lemma |6] says that, in essence, for an optimum rate 
communication F, 

- log ||F|| ^-[H (F|X") + H (F|y")] . 
n n 



Lemma 7. (A General Decomposition) For a CR J recover- 
able from an interactive communication F we have 

nI{X A Y) 

= / (X" A r" I J, F) + H{J, F) - iJ (F I X") 

-H{¥\ r") ~H{J\ T\ F)-H{J I y", F) . 

(12) 

Lemma |5] is a special case of ||6l Lemma B.l] (also, see 
lfT2l ). The proofs of Lemma |6] and Lemma [T] are given in 
the Appendix. 

Note that a simplification of (fT2l i gives 

"^ / (X" A y" I J, F) + 



I{X AY) < 



H{J, F)-H{F\ X") -H{F\ y") 



(13) 



If J is an e-CR recoverable from F, Fano's inequality implies 

- \H{J I X", F) + H{J I F", F)l < 2e log | A"] |y | + 2h{e) 
n 

= (5(e), say, (14) 

where h{e) ~ — eloge — (1 — e)log(l — e), and S{e) — > as 
e -^ 0. Combining ( fT2t and ilAi we get 



i{x A r) > - 

n 



/(X"Ay" I J,F) + H(J,F) 



-H{F\ X") - i/ (F I y^ 



(JW, (15) 



and further, by (fTOl i. 



/(X A y) > - [/ (X" A y" I J, F) + iJ( J, F) - H{F)] 
n 

~S{e). (16) 

A. Proof of Theorem \3\ 

In this section we give a proof for (|2). The proof of (|8) then 
follows upon taking limit r — > cxd on both sides of (ITJ. The 
proof of (|7]i follows from claims 1-3 below. In particular, the 
proofs of claims 1-3 establish a structural equivalence between 
a maximum rate SK and an SK of rate « —H{J \ F) extracted 
from a CR J recoverable from F such that L = (J, F) satisfies 
©. 

Claim 1: R^.^ > CmX;Y) ~ I{X A Y). 
Proof. By the definition of Rcj, for every < e < 1 there 
exists, for some n > 1, an r-interactive communication F of 
rate 



Since (J, F) satisfies 

i/(X"Ay" I J,F)<e<e + 5{e), 
n 

the inequaUty ( fTSl l. along with the fact that (e + d{e)) — > 
as e — !■ 0, implies that I{X A y) + Kj^j is an achievable r- 
interactive CI rate; hence, CIKX^Y) < I{X AY) + W^j. 



Claim 2: Rsk > Rci- 

Proof. Using the definition of R^g^, for < e < 1 there 
exists, for some n > 1, an r-interactive communication F of 
rate — log ||F|| < Rg^ + e, and an e-SK K recoverable from 
F of rate 

-H{K)>I{X AY)~e. (19) 

n 

By choosing J = A' in ( fT6l ) and rearranging the terms we get, 

-I (X" A y" I A', F) < I{X AY)- -H{K \ F) + (5(e). 
n n 

Next, from {l/n)I{K A F) < e, we have 

-/ (X" A y" I if, F) < I(X AY)- -H(K) + € + S(e) 
n n 

<2e + (5(e), 

where the last inequality follows from ([19}. Since (2e + 
(5(e)) ^> as e — >• 0, Rgj^ is an achievable 7--interactive 
communication rate for C/f , and thus, Rgj^ > Rqj- 

Claim 3: Wsk < CIHX; Y) - I{X A Y). 
Proof. For < e < 1, let J be an e-CR recoverable from 
an r-interactive communication F, with 

-H{J,F)<CmX;Y) + e, (20) 

n 

such that L ~ (J, F) satisfies (O, and so, by ( fTST l. 

-[H{F \X'') + H(F \Y")] 
n 

< -H{J,F)-I{X AY) + e 
n 

< CmX; Y) - I{X A y) + 2e. 



(21) 



-log||F||<i?^ 



CI 



(17) 



and J, an e-CR recoverable from F, such that L = (J, F) 
satisfies (|3]l. It follows upon rearranging the terms in (IT&i that 

-H{J, F) < I{X AY) + -H{F) + (5(e), 
n n 

which with ( fTTI i gives 

-H{J,F)<I{X AY)+Rlj + e + 6{e). (18) 

n 



To prove the assertion in claim 3, we show that for some 
N > 1 there exists A(e)-SK K = K{X^,Y^) of rate 

-logllA'll >/(XAy)-A(e) 
n 

recoverable from an r-interactive communication F" = 

F"(X^,y^) of rate 

^ log ||F"|| < ^ [HiF I X") + H{F I y")] + A(e) - 2e, 

(22) 

where A(e) -> as e ^ 0. Then (l22|, along with (l2T]i, would 
yield 

1 



N 



log ||F"|| < CIUX; Y) - I{X AY) + A(e), (23) 



so that CIl'{X; Y) — I{X A Y) is an achievable r-interactive 
communication rate for SK capacity, thereby establishing the 
claim. 



It remains to find K and F" as above. To that end, let J 
be recovered as Ji = Ji(X",F) and J2 ^ J2(F",F) by 
terminals X and y, respectively, i.e., 

P ( J = Ji = J2) > 1 - e. 

Further, for fc > 1, let 



and 



Ju - Ji [X'^li-i)+i,'Pij , 



J-ii - J2 (y„™_i)^i,F 



1 < i < fc, 



where F,=f(x^^,_,)+,,1;^^_,)_,, 



For odd r, we find 

an r-interactive communication F" such that (j^jF*^) is a 
e-CR recoverable from F", for all k sufficiently large; the 
the SK K will be chosen to be a function of (jfjF'^) 
of appropriate rate. The proof for even r is similar and is 
obtained by interchanging the roles of Ji and J2. In particular, 
by Lemma |6] for all k sufficiently large there exists an ?-- 
interactive communication F' such that F*"' is e-CR recoverable 
from F' of rate given by ( fTTT i. Next, from Fano's inequality 

-max{if(J| Ji);i7(Ji | J2)} < e log 1^-11^1 + /i(e). (24) 
n 

By the Slepian-Wolf theorem ifTSl there exists a mapping / 

of J^ of rate 



ilogl|/l|<H(Ji| J2)+ne, 



(25) 



such that 

Ji is e-recoverable from (/ ( J^') , J2 ) , (26) 

for all k sufficiently large. It follows from (l24l . (l25t that 



— log 11/11 <e + 6log|;f 113^1+ Me). (27) 

nk 

For N ~ nk, we define the r-interactive communication F" — 

F"(X^,y^) as 

F-' ^Fl, l<i<r-l, 
F;:^F;,f{J^), t^r, 

Thus, ( Jf , F'^) is 2e-CR recoverable from F", where, by (E) 
and ( |27] i. the rate of communication F" is bounded by 
1 



nk 



log||F"|| 



<-[H (F\X") + H (F|r")] + 2e + elog |A'||:y| + h{e). 
n 

(28) 

Finally, to construct the SK K = isr(j^,F'^), using the 
corollary of Balanced Coloring Lemma in ||5] Lemma B.3], 
with 

[/= (Ji,F), F = 0, n = A:, .9 = F', 

we get from (|28] | that there exists a function A' of jf , F*^ such 
that 

^ log 11^11 

>if(^)-ilog||F"|| 

> H{Ji,F) - H{F I X") - H{F I y") 

-n(2e + elog|Ar||3^|+/i(e)), (29) 



/(XAF') < cxp(-cA:), 



where c > 0, for all sufficiently large k. We get from ( |29] l and 
(IT3I 1 that the rate of /v is bounded below as follows: 



4-log||/^l| >I(XAY)--I(X''AY" I Ji,F) 
n/c n 



2e- elog 1^-113^1-/1(6). 



(30) 



Observe that 



/(X" A F" I J, F) = I{JuX'' A y" I J, F) 
>/(X"Ay" I J,Ji,F) 
>/(X"Ay" I Ji,F)-H{J\ Ji), 

which along with jTM . and the fact that L = (J, F) satisfies 
(|3]l, yields 



1 



/(X"Ay" I Ji,F)<e + elog 1^-113^1 +/i(e). (31) 



Upon combining ( l30l ) and dJTT i we get, 
1 



ifc 



^og\\K\\ > /(X A r)-3e-2e log 1^-113^1 -2h{e). 



Thus, for A(e) = 4e + 2e log | A^j |3^| + 2/i(e) isT is a A(e)-SK 
of rate {l/nk)log\\K\\ > I{XAY)-A{e), recoverable from 
r-interactive communication F", which with (l28T l, completes 
the proof. D 

B. Proof of Theorem |4] 

Achievability. Consider rvs Ui,...,Ur satisfying conditions 
(P1)-(P3) in the statement of Theorem|4] It suffices to show for 
every < e < 1, for some n > 1, there exists an r-interactive 
communication F, and e-CR J recoverable from F, such that 

I{X, y A [/'■)- e < -H{J, F) < I{X, r A [/'■) + e, (32) 
n 

and 

-H{F)<I{X,Y AU'')-I{X AY)+e, (33) 
n 

since from (fTSI l. (|32] | and (l33T l. we have 

-/(X" AF" I J,F) 
n 

< -H(F) - -H(J, F) + I(X AY) + 5(e) 
77. 77. 



<2e + (5(e). 
We show below that 



(34) 



I{X,Y AU'')~I{X AY) 

L(r-1)/2J 

= J2 I{XA C/2.+1 I Y, U^') 

i=0 

Lr/2J 

+ J2 I{YAU2^\X,U^'~^). 



(35) 



Thus, the proof will be completed upon showing that there {1, ...,n}. Define rvs U^ as follows: 
exists an e-CR J, recoverable from F of rate ,p , 

L(r-i)/2j Ui = F.i, 2<i <r, 

^H{F) < Y. ^(^ ^ t^2.+i I Y, C/2') C/ = / (^^ -^i)' ^ °'1'1' 

lT/Sj '■"1(^.,^2), reven. 

+ y^ I{Y A U2i I -'f, t/'^*^^) + e, (36) We complete the proof for odd r; the proof for even r can 

4=1 be completed similarly. It was shown by Kaspi ifTOl equations 

(3.10)-(3.13)]that 

such that ( J, F) satisfies (l32T i. For r = 2, such a construction ,, ,^ ^^,, ,^ „ , , , ,„ , 

was given by Ahlswede-Csiszar [/J Theorem 4.4]. (In their ^ ' ' — — lv // jj 

construction, F was additionally a function of J.) The exten- C^2j ^^ Yt, U '' -^ Xt, 1 < « < [r/2\ . 

sion of their construction to a general r is straightforward, and » , , , ^ jttt. .u . 

° ° Next, note from (I31l l that 

IS relegated to the appendix. 

It remains to prove (|35). Note e + e log | A:"! |3^| + h{e) 

> -IIX'' AY" I Ji,F) 

L(r-1)/2J n ' 



1=0 7J ^-^ 

- E I{YAU,.\X,U'^-') > i^/(X,Ay. |x-\y,!ti,Ji,F) 



-1 



L(r-1)/2J Lr-/2J 

= Y. ^(^ ^ ^24+1 I t/'') + E /(X A C/2. I C/2*-i). 

j=o i=i Similarly, it holds that 

(37) 



Further, from conditions (P1)-(P3) it follows that 



L(r-1)/2J L'-/2J 



/(Xt A Ft I [/'■). (39) 

e + elog \X\\y\ + hie) > I{Xt A r^+i | X^-\ Ji, F, T). 

(40) 

The entropy rate of (J, F) is now bounded as 
1 



Y ^Y ^ ^^24+1 I U^') + 5^ /(X A [72. I ?7'*-^) -HU F) 

'"-/(FAX) >ii/(Ji,F)-ii/(Ji|^) 

L(r-i)/2j ['■/aj ^ 1 



= 5: I{YAU,,^,\U'n+YHXAU,,\U'^-') >^H{.h,F)-elog\X\\y\^h{e) 

4 = 1 4=2 

+ I{X A U2 I f/i) + I{Y A [/i) - I{Y A X) 



i/(X", F" A Ji,F)- elog 1^-113^1-/1(6) 
L('-i)/2J Lr-Zaj =i?(XT,rT)--i/(X" I ^i,F) 



5^ /(y A [/2.+1 1 [/2'0 + E /(x A U2^ I t/2'-i) 



4=1 4=2 --H{Y''\X",Ji,F)-elog\X\\y\-h{e) 



+ I{X AU2 I t/i)-/(XAy I Ui) 
L(r-i)/2j ['■/aj 



^T-l 



^ H{Xt,Yt) - H{Xt \X'-\Ji,F,T) 

V /(yAC/2^+1 I U^')+ V I{X AU2; I ?7^'"^) -i^(5^T |X^-\yT\i,XT,XJ+i,Ji,F,T) 

4=2 -elog|A'||y|-/i(e) 

-/(XAy I Ui,U2) >I{XT,YTAU'')-e-2e\og\X\\y\ -2h{e), 



-I{X A y I [/'■) = 0. (38) 



Combining ^^ and ([381) we get ( [35] l. 

Converse. Let i? > be an achievable r-interactive CI rate. /? > —H{J, F) — e 



where the second inequality follows from Fano's inequality, 
and the last inequality follows from ( |40l i. Consequently, 



n 
> I{Xt, Yt a W) - 2(e + e log \X\\y\ + h{e)). (41) 



Then, for all < e < 1, for some n > 1, there exists an r- 

interactive communication F, and e-CR J recoverable from F, 

such that {l/n)H{J, F) < R + e and L = (J, F) satisfies ([3]). We now replace the rvs Ui, ..., Ur with those taking values in 

Let J be recovered as Ji = Ji(X",F) and J2 = J2(y",F) finites sets Ui,...,h(r, respectively, with Ui,...,Ur satisfying 

by terminals X and y, respectively, i.e., P ( J = Ji = J2) > the cardinality bounds in condition (iii). Similar bounds were 

1 — e. Further, let rv T be distributed uniformly over the set derived in the context of interactive function computation in 



lUn . For 1 < / < r, assume that rvs I4i, ...,l4i-i satisfy the 
cardinality bounds. We consider odd /; the steps for even I are 
similar. If the rv Ui does not satisfy the cardinality bound, from 
the Support Lemma lH Lemma 15.4], we can replace it with 
another rv Ui that takes less than or equal to | A"! Y[i=i l^d + 1 
values, while keeping the following quantities unchanged: 

PxtU'-^. HXtAYt I [/'■), and I{Xt,Yt A [/'■). 

Note that we have only altered P[/, in the joint pmf 
PxtYtU- = ^UiPxtU'-^\Ui^Yt\xu'-^- Hence, the Markov re- 
lations in (PI) remain unaltered. Furthermore, PxtYt = ^xv- 
Finally, since the set of pmfs on a finite alphabet is compact, 
and the choice of e above was arbitrary, it follows upon taking 
e -^ in ([39]l and (gB that there exists [/[ satisfying (P1)-(P3) 
such that 

R>I{X,Y AW), 



which completes the proof. 



D 



V. Can interaction reduce the communication 

RATE? 

It is well known that the SK capacity can be attained by 
using a simple one-way communication from terminal X to 
terminal y (or from y to X). Here we derive the minimum 
rate Rnj of such noninteractive communication using the 
expression for CIl{X;Y) in (|9|l. Since this expression has a 
double Markov structure, it can be simplified by the following 
observation (see ID Problem 16.25]): If rvs U,X,Y satisfy 



U ^X ^Y, X ^U ^Y, 



(42) 



then there exist functions / = f{U) and g = g{X) such that 
(i) F{f{U)=g{X)) = l- 
(ii) X ^ g{X) ^ Y. 
In particular, for rvs U, X, Y that satisfy ( |42] |. it follows from 
(i) above that 

liX, YAU) = IiXAU)> I{g{X) A f{U)) - H{g{X)). 

Turning to (|9]l, for rvs U'' with r odd, the observations 
above applied to the rvs X and Y conditioned on each 



realization C/ 

gi = .91 (X, [/- 



r-l 



,r-l 



implies that there exists a function 



such that 



x^g (x, v-^) , v-^ ^ y, 



(43) 



and 



I{X,Y A [/'■) > / (X, y A U'-^)+H {g {X, U""^) \ C/'-^) 

where rv U^~^ satisfies (PI), (P3). Similar observations hold 
for even r. Thus, for the minimization in (|9]l, conditioned on 
arbitrarily chosen rvs [/'"^ satisfying (PI), (P3), the rv Ur is 
selected as a sufficient statistic for Y given the observation X 
(sufficient statistic for X given the observation Y) when r is 
odd (r is even). Specifically, for r = 1, we have 

Cll{X-Y)= min i/(<?i(X)), (44) 

X^gi{X)^Y 



and 



Cll{Y-X)^ min H {g2{Y)) . (45) 

Y^g2(Y)^X 



The answer to the optimization problems in (l44l i and (|45) can 
be given explicitly. In fact, we specify next a minimal sufficient 
statistic for Y on the basis of X. Define an equivalence relation 
on X as follows: 

x^x' <=> Py|x {y\x)= 7y\x {y I x') , yey. (46) 

Let gl be the function corresponding to the equivalence classes 
of ^. We claim that gl is a minimal sufficient statistic for Y 
on the basis of X. This expression for the minimal sufficient 
statistic was also given in 19] Lemma 3.5(4)]. Specifically, 
X -^ gi{X) ^ Y since with gl{X) = c, say, we have 

PyIsUX) {y I c) 

= J2^Y^x\9Ux) {y,x I c) 
xex 

= Yl ^x\aUx){x\c)PY\x,gUx)(.y\x,c) 

= ^Y\x,gi(x) {y \x,c), V.T with gl{x) = c. 

Also, if gi{X) satisfies X -e~ gi{X) -e- Y then g* is a function 
of gi. To see this, let gi{x) — gi{x') = c for some x, x' G X. 
Then, 



Y\gt{X) 



{y I c) 



Y\X 



[y I x) == Y>Y\x {y\x'), y£ y, 



so that gl{x) = g*(.T'). Since gl is a minimal sufficient 
statistic for Y on the basis of X, it follows from ( l44l i that 

C/i(X;r) = F(.g*(^)), 
and similarly, with ,92 (^) defined analogously, 

ClliY;X)^Hig;{Y)). 

Therefore, from (|7), the minimum rate Rnj of a noninteractive 
communication for generating a maximum rate SK is given by 

Rni = min {i/ (gliX)) , F (.gi*(^))} - HX A Y). (47) 

From the expression for Rni, it is clear that the rate of 
noninteractive communication can be reduced by replacing 
X and Y with their respective minimal sufficient statistics 
gl{X) and g2{Y). Can the rate of communication required 
for generating an optimum rate SK be reduced by resorting 
to complex interactive communication protocols defined in 
Section |II]? To answer this question we must compare the 
expression for R^j with Rsk- Specifically, from Theorem 
|3] and the Corollary following it, interaction reduces the rate 
of communication iff, for some r > 1, 

CmX;Y) < mm{H {gliX)) ,H {gliX))} , (48) 

where gl and 92 ^re as in (|47| i: interaction does not help iff 

CI,{X A r) = min{iJ (g*(X)) , H {gl{X))} . 

Note that instead of comparing with CIl{X;Y) in (|48] l, we 
can also compare with CIl{Y;X). 

We shall explore this question here, and give an example 
where the answer is in the affirmative. In fact, we first show 
that interaction does not help in the case of binary symmetric 
sources. Then we give an example where interaction does help. 



A. Binary Symmetric Sources 

For binary rvs X and Y, we note a property of rvs U^ that 
satisfy the conditions (P1)-(P3) in Theorem |4] 

Lemma 8. Let X and Y be {0, 1} valued rvs with I{Xt\Y) ^ 
0. Then, for rvs Ui, ...,Ur that satisfy the conditions (P1)-(P3) 
in Theorem^ for every realization wi, ..., Ur ofUi, ..., Ur, one 
of the following holds: 

H{X I V = i/) = 0, or H{Y \ U'' = vJ') = 0. (49) 

Proof. Given a sequence ii^ , assume that 

H(X I [/'■ ^ u'') > and H{Y \ W = u"") > 0, 

which is equivalent to 

Px\u^{l\u'-)Px\uAO\u'-)>0 and 

Py|C/.(l|u'^)Py|C;.(0|M'^)>0. (50) 

We consider the case when r is even; the case of odd r is 
handled similarly. From the Markov conditions X -^ U'' -^ Y 
and X -e-Y, U^^^ -^s- Ur, we have 

Px,Y\u- {x,y I u'') 

= Px\uAx \u'-)PY\uAy\u'') 

= Px\YM--^ {x I y, w'-i) Py\W' {y \W), x,ye {0, 1}. 
Since Pyiw {y I "u^) > from ( fSOl l. we have 

Px\U'' {x I u'') = Px\Y,U''-^ {x I y, u''"^) , x,y e {0, 1}, 
which further implies 



if X and 1^ are independent, t{u'') ~ 0. Note that t is a 
stopping-time adapted to Ui,...,!/,.- Then, from ( fSTT i, ( |52] |, 
CIl{X\ Y) remains unchanged if we restrict the support of [/'' 
to sequences u^' with ui ~ (f> for all i > t(u''). Furthermore, 
the Markov condition (PI) implies that if for a sequence u^ , 
T = t{u^) is odd then 

Py|x,(7- (y I 2:,M^) = Pf|x,(7— 1 (y I x,?/^"^) , 

and so if 

Px\u^il\u'')Px\u^{0\u^)>0, 

it holds from the definition of r that 

Py\U-{1 |M")Py|[/. (0|u")>0, 

which is a contradiction. Therefore, we have H{X \ U"^ = 
u") = 0. Similarly, H{Y \ V = u") ^ G holds for even r. 
To summarize. 



CIl{X- Y) ^ mmI{X, YAU^), 



(53) 



Pxiy^c/-! {x\l,'> 



)^PxiY^U''-^{x\0,u^-'), 

a;e{0,l}. 

Hence, I{X AY \ U"-^ = u"-^) = 0. Noting from ^ that 

Px\u^-i (1 I w'--^) Px|a-i (0 I u--') > 0, 

we can do the same analysis as above, again for r — 1. Upon 
repeating this process r times we get I{X AY)~0, which 
is a contradiction. Therefore, either H{X \ U"^ ^ u^) = or 
H{Y I V = u'') = holds. D 

Note that 

CIl{X- Y) = H{X, Y) - m-AxHiX, Y \ [/''), 

where the max is taken over rvs [/'' as in Theorem|4] If H{X \ 
[/» = -u*) = 0, it follows that 

/ (X A r I [/' = u') = 0, and 
H{X, Y\W^ u\ Ul^^) = H{Y I U' = u\ t/f+i) 

< iJ(r I [/^ == m')- (51) 

Similarly, H{Y \ W = u') = implies 

I{X AY \W ^u') ^0, and 
H{X,Y I W = u\ U[+i) < H{X I U' = v^). (52) 

For a sequence u'' with P[/r (u'') > 0, let t{u'^) be the 
minimum value of i such that 

H{X I t/' = u') = or H{Y \ W = ?i*) = 0; 



where U^ are rvs satisfying (P1)-(P3), and r is the stopping- 
time defined above. 

We show next that for binary symmetric sources, interaction 
can never reduce the rate of communication for optimum rate 
SK generation. In fact, we conjecture that for any binary rvs 
X, Y, Rxi — RsK- 

Theorem 9. Let X and Y be {0, l}-valued rvs, with 

P {X = 0,Y = 0) =¥{X = l,Y = 1) = -{I -S), 

F{X = Q,Y = l)=F{X = 1,Y = 0) = ^S, < S < -. 

(54) 



Then, 



CIi{X AY)= mm{H{Xy,H{Y)}, 



i.e., interaction does not help to reduce the communication 
required for optimum rate SK generation. 

Remark. As a consequence of Theorem |9] for sources with 
joint distribution as in ( |54] |. the second inequality in ^ can be 
strict. Specifically, it was noted by Wyner (see the discussion 
following equation (1.19) in |fT6ll ) that for binary symmetric 
sources, CIw{X AY) < 1. From Theorem |9] we have 

CIi{X AY) =mm{H{X);H{Y)} = 1. 

Thus, for such sources, Clwi^ AY) < CIi{X A Y). 

Proof. Denote by U^ the following set of stopped sequences 

inZ^'': 

For i < r, for a sequence u^ G U^ the stopped sequence 

u' G U^ if: 

H {X I W = u^) >0,H {Y \ W = u^) > 0, V j < i, and 
H{X \W ^u') ^Ooi H{Y \W = u') = 0. 

For i e {0, 1}, define the following subsets of Uq: 

U^ ^ {u'' €U^:t is odd,Px\u- (i I w^) = 1} , 
Ul = {u'^ eU^ :t is even, Py\u-- (« I u^) = l} • 



By their definition the sets U^ , U^ , U^ , and UY are disjoint, 
whereby we have 

=J2[^u^i^n+^ur{ur)]^i. (55) 

i=0 

For u^ G Uq, denote by p{u'^) the probability Pj/t (u"^). 
Further, for w^ G U^ \JU^ , denote by W^^ : X ->■ y the 
stochastic matrix corresponding to Py\x,u^ (" I ■i'"^)^ ^nd for 
u'^ e UqIJUY, denote by T"" : y ^ X the stochastic 
matrix corresponding to Px\y.u^ (' I ': W^). With this notation, 
the following holds; 

= Px,Y [hi) 



u^ eui^ 



u^ eUf 



e{0,l}, (56) 



since the sets U^ ,U^ ,14^ ,UY are disjoint. Upon adding 
for i = 0, 1, we get 



E 



J2 Piunw-^ii\i)+ J2 Pi^lT-^i^l^) 



u^euf 



u^ &Uf 



On adding ( l57b and ( l58b we get 

;i(<S)=.g((5)+.g(l-(5) 
1 
>E E P{ulh(w-\^\^)) + 

where h is the binary entropy function. Note that the right side 
above equals H{X,Y | C/'^), which yields 

h{S) = m&x{H{X I Y); H{Y \ X)} > H{X, Y \ W). 

Since rvs If above were arbitrary, we have from (l5Ji , 

C/[(X;y) > H{X,Y) - max{H{X \ Y);H(Y \ X}) 
= \imi{H{X)]H{Y)}. 

Combining this with (|6]), we obtain 

CIl{X- Y) = mm{H{Xy, H{Y)}. 

D 

B. An example where interaction does help 

Consider rvs X and Y with A" = 3^ = {0, 1, 2}, and with 
joint pmf: 



= (l-<5). 
Furthermore, from (|55t we get 



a 


a 


a 


h 


a 


a 


a 


c 


a 



i^E E p("')+ E p("')- 

Therefore, since the function g{z) = — zlogz is concave for 
< z < 1, the Jensen's inequality yields 



ff(l-^)>E E P{nn9[w-\z\h 



where a, 6, c are nonnegative, 7a + &+c = 1, and c^ a, which 
holds iff 6 ^ 1 — 8a. Assume that 



2a>b> a. 



(59) 



^ p{u^)g{T-\i\i)) (57) 



From ( l48l l. to show that interaction helps, it suffices to find 
rvs Ui, ..., Ur satisfying (P1)-(P3) such that 

I {X,Y AUi, ...,Ur) < mm{H (gliX)) , H {g;{Y))} , 

(60) 



u^&U} 



Similarly, using 

7:0 = ?x,y[i,]) 



where gl and 32 ^rc ^s in ( |47] |. From (l46t . ^^(a:;) = 51(2;') iff 

(61) 



Pr,x(y,x) _ Px(.t)_ 



5^ pK)(l-r'^(j|j)), *^.?,z,je{0,l}, 



M^ GW/ 



we get 



VY,x{y,x') ?x{x'Y 

i.e., the ratio -^■'^, ' ,. does not depend on y. Therefore, for 
the pmf above, gl{X) and .92 (^) ^6 equivalent to X and F, 
respectively. Thus, 

mm{H{gl{X)),H{g;{Y))} = mm{H{Xy,H{Y)}, 

where H{X) = H{Y) for the given pmf. 

Next, let Ui = fi{X), U2 = /2(r,/i(X)), where /i and 
/2 are given below: 



5W>E E Piul9[l~W-^{^\^) 



/i(^) 



1, x = 2, 

2, x==0, 1, 



^ Piu^)g(l-T^^{z\z)) (58) 



«^ ew/ 



/2(j/,l) = 0,Vz/ e {0,1,2}, and /2(y,2) = 



1, y = o, 

2, y = i,2. 
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Clearly, Ui and U2 satisfy (PI) and (P2). For (P3), note that 
if iUi,U2) = (1,0), then X = 2, and if iUi,U2) = (2,1), 
then r = 0. Finally, if (t/i,t/2) = (2,2), then X G {0,1} 
and Y G {1,2}, implying 



4a 



1 



= -, V(.T,y)e{0,l}x{l,2}. 

Therefore, I{X AY \ Ui, U2) = 0, and so [/i, t/2 satisfy (P3). 
We show that (|60] | holds for this choice of Ui,U2- Specifically, 
I{X,Y AUi, U2) = H ([/i, t/2), and the following holds: 

H{Y)-H{Ui,U2) 
= H{X)-H{Ui,U2) 
= H {X\Ui) - H {U2\Ui) 



= P(/i(^) 

= (5a + b) 
= (5a + b) 



H{X\h{X)^2) 



-i/(/2(2,y)|/i(X) = 2) 

h{Px\MX)m))^h{PYiMx)m))] 
3a \ , / a + b 



Then, from (jsg. 



5a + 6 



a + 6 
5a + fe 



-h 



5a + b 



< 



3a 1 

5a + 6 "^ 2' 



which implies (l60b for Ui,U2- 
VI. Sufficient statistics and common information 

QUANTITIES 

In this work we encountered three CI quantities: Shannon's 
mutual information I{X A Y), Wyner's CI CIw{X A Y), 
and interactive CI CIi{X AY). In fact, the first notion of 
CI was given by Gacs and Korner in the seminal work fS]. 
In particular, they specified the maximal common function of 
X and Y, denoted here as mcf{X,Y), such that any other 
common function of X and F is a function of mcf{X,Y); 
the Gacs-Korner CI is given by H{mcf{X, Y)). The following 
inequality ensues (see lH), lfT6l . and inequality (|6]l): 

iJ(incf(X,r)) <I{XAY) <CIwiXAY) <Ch{X AY). 

Since any good notion of CI between rvs X and Y measures 
the correlation between X and Y, it is reasonable to expect the 
CI to remain unchanged if X and Y are replaced by their re- 
spective sufficient statistics. The following theorem establishes 
this for the quantities i7(mcf(X, r)), /(XAF), (^/^(XAr), 
and H{mcf{X,Y)). 

Theorem 10. For rvs X and Y, let functions gi of X and 32 
of Y be such that X -e- gi{X) -e- Y and X -e- ,92 (^) -^ Y. 
Then the following relations hold: 

H{mcf{X,Y))=H{mcf{gi{X),g2{Y))), 

I{XAY)^I{g,{X)Ag2{Y)), 
ClwiX AY)=CI {gi{X) A g2{Y)) , 

CmX;Y)^Ci:{gi{X);g2{Y)), r > 1, 
Ch{X A y) = Ch (giiX) A g2iY)) . 



Remark, (i) Theorem [TO] implies that the minimum rate of 
communication for generating a maximum rate secret key 
remains unchanged if X and Y are replaced by gi{X) and 
92{Y) as above, respectively. 

(ii) Note that gi{X) and g2{y) above are, respectively, func- 
tions of gl{X) and g2iY) defined through ( |46] |. 

Proof. First note that 

liX AY)=I igi{X) AY) = I (51 (X) A 32(1^)) • (62) 



Next, consider the interactive CI. From (I62t . any protocol 
that generates an optimum rate SK for the sources gi{X) and 
g2{Y) also generates an optimum rate SK for the sources 
X and Y. Thus, the minimum communication rate for prior 
protocols is bounded below by the minimum communication 
rate for the latter protocols, so that by Theorem |3] 

Ci: {g,{X)-g2{Y)) - I {g,{X) A 52 (F)) 
> CIl {X- Y)-I{XAY), 

which, by ( |62] |. is 

CIl{g,{X)-g2{Y))>CIl{X-Y). (63) 

In fact, ( l63l l holds with equality: We claim that any choice of 
rvs U^ that satisfy (P1)-(P3) also satisfy the following Markov 
relations: 

C/2,+1 ^ .91 (X), C/2^ ^ g2{Y), < i < L(r - 1)/2J, 



U2^^g2iY),U 



2i-l 



-giiX), l<t<lr/2\, 



(64) 



giiX)^U'^^g2iY). 

It follows that 

CIl {g^{X)-g2{Y)) < I {g,{X),g2{Y) A U^ 
<I{X,Y AW), 

and consequently, 

CIl{g^{X)-g2{Y))<CIl{X-Y). 
Thus, by ^, 

CIl (5i(X);.92(r)) = CIl {X-Y). (65) 

Taking the limit r ^> 00 we get 

CU igi{X) A g2{Y)) = CI, (XAY). 

It remains to establish ( l64] i: instead, using induction we estab- 
lish the following stronger Markov relations: For I < i < r, 

U, H^ .91 (X), t/'-i -^ r, i odd, 

U,^g2{Y),U'-^ ^X, ieven, 

X -^ gi{X), W ^Y and X -^ .92(1"), U' -^ Y. (66) 

Clearly, (|66] | implies the first two Markov relations in (l64l . 
The last Markov chain in ( l64l i follows upon observing 

= I{XAY\U'')>I {gi{X) A g2{Y) \ W) . 

To see that (l66ll holds for i = 1 note that 



I{XAY\gi{X),Ui) 
<I{XAY I .91 (X)) +I{UiAY I gi{X),X) = 0, 
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and 

I{X^Y\g2{Y),U^) 
<I{XAY I g2{Y)) + / (C/i A Y, g^iY) | X) = 0. 

Next, assume that ( l66b holds for an even i. Then, from (PI) 
we get: 

/ (y A c/,+1 1 X, w) = 

^l{YAU,+i \X,gi{X),W)^0 

^I [Y A X, C/,+1 I gi{X), W)^I{YAX\ .gi(X), W) = 0, 

where the last equality follows from (l66T l. From the last 
inequality above we have 

U^+i^gi{X),W ^Y and X ^ gi{X),W+'^ ^Y. 

Furthermore, it also follows from (|66) that 

/ (X A y I g2{Y), C/'+i) < / {X, U,+i A Y \ g^iY), W) 
^l{U,+iAY\g2iY),X,W) 
< I ([/,+i AY \X,W) = 0, 

where the last equality follows from (PI). Thus, we have 

X ~<^ g2iY),U'+^ -e^r, 

establishing the validity of ( |66] | for i + 1. The proof of (|64] | 
can be completed by induction by using a similar argument 
for odd i. 

Next, we consider the Gacs-Korner CI. Note that any 
common function of gi{X) and 52 (y) is also a common 
function of X and y. Consequently, 

H(mcf(X,y)) >i7(mcf(gi(X),g2(y))). (67) 

For the reverse inequality, observe that for an rv U such that 
H{U\Y) = H{U\X) = we have 

U ^X ^gi{X)^Y. 

Thus, H{U\gi{X)) < H{U\Y) = 0, and similarly, 
H {U\g2(Y)) = 0. In particular, it holds that 

H (mcf (X, Y)\gi{X)) = H (mcf (X, y)|.92(y)) = 0, 

and so, 

H(mcf{X,Y)) < H(mcf{gi{X),g2{Y))), 
which along with ( |67] | yields 

Himci{X,Y)) = i/(mcf(,9i(X),g2(y))). 

Finally, we consider Wyner's CI and claim that this, too, 
remains unchanged upon replacing the sources with their 
respective sufficient statistics (for the other source). It suffices 
to show that 

ClwiX AY) = ClwigiX) AY), 

for a function g such that X -e- g{X) -e- Y. Consider an rv 
W for which X -e^ W -^ Y is satisfied. We have 

= liX AY \W)>I {g{X) A y I 1^) . 



It follows from dUi that 

CIw {X AY)> CIw {giX) A Y) . (68) 

On the other hand, for an rv L = L (g" {X"') , y") we have 

-/ (X" A y" I L) =: -/ (5" (X") A y" I L) , 

n n 

since 

J (^« A y" I L, g" {X")) <I{X" A y", L \ g" (X")) 

= / (X" A y" I g" (X")) = 0. 

Thus, from the definition of ClwidiX) A Y) we get 

CIw{X AY)< ClwigiX) A y), 

so that, by ( |68] l, 

CIw{X AY) = ClwigiX) AY). 

D 

VII. Discussion 

A. Local Randomization 

Although independent local randomization was not allowed 
in our formulation, our main result characterizing Rsk holds 
even when such randomization is available. Consider a model 
where terminals X and y, in additional to their respective 
observations X" and y", have access to finite-valueqj rvs 
Ti and T2, respectively. The rvs Ti, T2, and (X",y") are 
mutually independent. The SK capacity is defined as before, 
with X" and Y" now replaced by (X",Ti) and (y",T2), 
respectively. It is known ifTSl . ^ that even with randomization 
the SK capacity equals /(X A Y). For this model, denote 
the minimum rate of r-interactive communication required to 
generate an SK of rate /(X A Y) by ^^^. 



Lemma 11. For r > 1, 

fyr ur 

~ ~ r 

To see this, we define quantities Rqj and C/j analogously 
to R'sj^ and C/[, with X" and Y" replaced by (X", Ti) and 
(y™, T2), respectively. Note that this substitution is made even 
in condition (O, i.e., the CR J and the communication F now 
are required to satisfy: 



-/ iX'\ Ti A y", T2 I J, F) < e. 



(69) 



We observe that ^ still holds, with iX'\Ti) and (y",r2) 
replacing, respectively, X" and Y" on the right-side. There- 
fore, the proof of Theorem |3] is valid, and we get: 



Rhi = R'sK = CI, - HX A Y). 



(70) 



By its definition R'a < Rci, since L = (J, F) = L(X", y") 
satisfying ^ will meet ( |69] ) as well. We claim that R^fjj > 
Rqj, which by dTOb and Theorem [3] implies Lemma [TTI 



^The cardinalities of the range spaces of Ti and T2 are allowed to be at 
most exponential in n. 
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Fig. 1. Minimum rate of communication Rsk for optimum rate SK 
generation 



Indeed, consider CR J recoverable from F such that (J, F) 
attain BJf-,j. Then, the condition (|69] | gives 

-/(x"Ay" I j,F,ri,T2)«o. 

n 
So, there exist t\ , ^2 such that conditioned on Ti = ti,T2 = ^2 
the CR J is still recoverable from F, and 

-/(x"Ar"| j,F,ri = ti,T2 = f2)«o. 

n 

Thus, with Ti = ti, r2 = ^2 fixed, (J, F) constitutes a feasible 
choice in the definition of R^j- Since the number of values 
taken by F can only decrease upon fixing Ti = ii,T2 = 
t2, we get Rqj > Rqi- Therefore, the availability of local 
randomization does not decrease the rate of communication 
required for generating an optimum rate SK. 



B. Less-than-optimum rate SKs 

SK generation is linked intrinsically to the efficient genera- 
tion of CR. For /9 > 0, a rate i? > is an achievable CR rate 
for p if for every < e < 1 there exists, for some n > 1, an 
e-CR L with 

-H{L) >R-e, 
n 

recoverable from an r-interactive communication F, for arbi- 
trary r, of rate 

-H{¥)<p + e- 
n 

the maximum achievable CR rate for p is denoted by CR{p). 
Similarly, denote by C{p) the maximum rate of an SK that 
can be generated using a communication as above. It can be 
shown in a straightforward manner that 



C{p) = CR{p) - p. 



(71) 



The graph of CR as a function of p is plotted in Fig.[T] CR{p) 
is an increasing and a concave function of p, as seen from a 
simple time-sharing argument. Since Rsk is the minimum 



rate of communication required to generate a maximum rate 
SK, CR{p) - p = I{X A Y) for p > Rsk- Thus, our 
results characterize the graph of CR{p) for all p > Rsk- 
The quantity Rsk is the minimum value of p for which the 
slope of CR{p) is 1; CR{Rsk) is equal to the interactive 
common information Cli {XAY). Furthermore, from the proof 
of Theorem[3] a CR L that satisfies (O must yield an optimum 
rate SK. Thus, any CR recoverable from a communication of 
rate less than Rsk cannot satisfy (O. A characterization of 
CR{p) for p < Rsk is central to the characterization of C{p), 
and this, along with a single-letter characterization of Rsk, 
remains an interesting open problem. 

Appendix 

Proof of Lemma |6} 

From the Slepian-Wolf theorem ifTSl . there exist mappings 
fi, ..., fr of F^, ..., F^, respectively, of rates 

1 TIE 

-\og\\f2,+i\\<H{F2^+i\Y'\F,,...,F2^) + -^ 



1 TI f 

^logl|/2.|| < H{F2. I X",^i,...,F2,_i) + - 



0<z< [{r-l)/2\, 
l<z<Lr/2j, 



such that 



7k 



Fn],-! is - — recoverable from 



2i+l 



2r 



{f2^+l{F^,+^),Y^,F,\...,F,\),0< ^<[ir- 1)/2J, 



Fo, is recoverable from 

(/2,(F2'),^^,^f,-,i^2Vl), 1<*< k/2j, 

for all k sufficiently large. Thus, the communication F' given 
by Fl = fi (F^), 1 < i < r constitutes the required 
communication of rate 

■\ log IIF'II <-[H (F|X") + H (F|r")] + e. 
nk n 



D 



Proof of Lemma [7} 

For T = T (X", y") we have. 



nI{X AY) 
= H (X", F") - H (X" I y") - iJ (F" I X") 

= i? (X", r" I T) - i7 (X" I y", T) - ij (y" | x", r) 

+ H{T) -H{T\ X") -H{T\ y") 
= / (X" A y" I T) + H{T) -H{T\ X") -H{T\ y") . 

Lemma |7] follows upon choosing T ^ J,F. D 



Proof of ( 1521 ) and i 

It remains to prove that there exists e-CR J, recoverable 
from F such that J, F satisfy ( |32] ) and ( |36] ). We provide a CR 
generation scheme with r stages. For 1 < A: < ?', denote by 
£k the error event in the fcth stage (defined below recursively 
in terms of £k-i), and by Eq the negligible probability event 
corresponding to X",y" not being Pxy-typical. 

Consider 1 < k < r, k odd. For brevity, denote by V the 
rvs C/*''^^ and by U the rv Uk', for A: = 1, y is taken to 



be a constant. Suppose that conditioned on £j:_i terminals X 
and 3^ observe, respectively, sequences x G A"" and y G y^\ 
as well as a common sequence v 6 V" such that (v, x, y) 
are jointly Pyxy-typical. For S > 0, generate at random 
exp [■n{I{X, Y AU \V) + S)] sequences u G U" that are 
jointly P[/y-typical with v, denoted by u^j, I < i < Ni, 
1 < .? < N2, where 

Ni = exp [n {I{X AU \Y,V) + 36)] , 
iV2 = exp [n {I{Y AU \V)- 2d)] . 

The sequences u^ are generated independently for different 
indices ij. Denote by i('^')(v,x) a sequence u^, 1 < i < A^i, 
1 < J < ^2, that is jointly P[/yx-typical with (v, x) (if there 
exist more than one such sequences, choose any of them). 
The error event when no such sequence is found is denoted 
by £ki', this happens with probability vanishing to doubly 
exponentially in n. The communication Ffc(v, x) is defined to 
equal the first index i of u^ = i'^'^')(v,x). Upon observing 
Ffc(v,x) = i, the terminal y computes £3 (v,y,i) as the 
unique sequence in {u^j, 1 < j < N2}, that is jointly typical 
with (v,y). If no such sequence is found or if several such 
sequences are found an error event £k2 occurs. Clearly, the 
rate of communication Fk is bounded above by 

- logTVi ^I(X AU \Y,V) + 3d 
n 

= /(XA(7fc |r,[/'=-i) + 35, (Al) 

and also, for large n, 

-H(L^^^ ) < - log(l + NiN2)< I(X, Y AU\V) + 2S 
n n 

= liX, Y AUk\ Y, U''-^) + 2(5. (A2) 

Denote by £kz the event i^L^^'' (v, x), v, x, y) not being jointly 
-Pi/yxy-typical. The error event £k is defined as £k = £k-i U 
£ki U £k2 U £k3- Then, conditioned on £^ the terminals share 
sequences (u^j, v) that are jointly typical with (x, y). In the 
next stage fc + 1, the sequence (uy, v) plays the role of the 
sequence v. The scheme for stages with even k is defined 
analogously with roles of X and 3^ interchanged. We claim 
that i'^\ ...,L'^^'> constitutes the required CR along with the 
communication F = Fi, ..., Fk- Then, (|36] | follows from (lAlb . 
and the second inequality in ( |32] | follows from (IA2b . Moreover, 
for every realization Ui,...,Ur of L''^\ . . . , L^'^\ with E = Ig,. 
we have, 

lW PM- 



Ui,...,Ur \ E = 0j 

< IP({(x,y) : (ui,...,u,-,x,y) are jointly Pu-xy typical}) 

< exp [-n{I{X, Y AU'')- S)] , 
for n large, which further yields 

-iJ(L(i)...LM \E^0)> I{X, Y AU'')- S. 
n 

Therefore, 

n 

> ii/(i(i). ..£('■) I ^ = 0)-P(f^) log 1^-113^1 
n 

> I{X, Y AU'')-5-V {£r) log I A'l |3^|. 



Thus, the claim will follow upon showing that P(£'r) ^ as 
ri — > 00. In particular, it remains to show that V {£^2) — > 
and P(ffc3) — > 0, fc = l,...,r, as n — > cio. As before, we 
show this for odd fc and the proof for even fc follows mutatis 
mutandis. To that end, note first that for any jointly Puvx- 
typical (u, v,x), the set of y e 3^" such that (u, v,x, y) are 
jointly typical with (u, v, x) has conditional probability close 
to 1 conditioned on f7" = u, V" — v, X" = x, and so by 
the Markov relation Y -o- V, X -e- U, also conditioned on 
V"^ = v,X" = X. Upon choosing u = i'^'^)(v,x) in the 
argument above, we get ¥ {£k2) -^ 0. Finally, we show that 
P (ffcs) will be small, for large probability choices of the ran- 
dom codebook {uy}. Specifically, for fixed typical sequences 
(v, X, y), the probability P {£k3 \ V" = v, X" = x, F" = y) 
is bounded above exactly as in [2] equation (4.15)]: 

P(ffc3|V^"-v,X"=x,r" = y) 

Ni N2 N2 / 

< ^ ^ ^ P f (u,j , V, x) jointly Pj/yx -typical, 
i=i j=i i=i,i^j ^ 

(uii,v, Ui;) jointly Pc/yy-typical 

< NiNl exp[-n(/(X AU \V) + I{X AU \V) + o{n))] 

< cxp[— 71(5 + o{n)], 

for all n sufficiently large. Note that the probability distribution 
in the calculation above comes from codebook generation, 
and in particular, the second inequality above uses the fact 
that Uii and u^ are independently selected for I ^ j- Thus, 
P {£k3 I £k2 ) -^ for an appropriately chosen codebook, 
which completes the proof. D 
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